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(54) User input device for an interactive display system 

m\ The user input device comprises a body (30, 32) on which are mounted first input means, eg. tracker ball (36) or 
KtK of a user relative to the body are translated into a first control signal variable in at least 

input means (64) eg. of the "MOUSE" type by which movement by the user of the body in two 
dTmeS^s against which the body rests is translated into a second control signa variable ,n two 

d imeSs ioSs The dev ce enables the generation of two independent, multi-dimensional control signals, avoiding or reducing 
^^^wSita^ the control for example of 3-D image synthesis display apparatus by means of known user input 
dS^TiS^npiit means may be surrounded by a bezel (46, Figs 5, 6) which is rotatable by the user's fingers to 
generate further control signals. 

Fig 2. 
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At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy. 
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Fig. 5. 
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DESCRIPTION 

USER INPUT DEVICE AND DISPLAY SYSTEM INCLUDING SUCH A DEVICE 
The invention relates to a user input device for an 
interactive display apparatus, the device comprising a body on 
which are mounted first input means by which hand movements of a 
user relative to the body are translated into a first control 
signal variable in at least two dimensions. 

The invention further relates to an interactive display 

system comprising: 

first input means comprising a user input device as set 
forth in the preceding paragraph, second input means and 

signalling means; 

three-dimensional (3-D) modelling means for defining a 

set of objects in a 3-D object space; 

transformation- means for generating representations of 

the objects in a viewer space; 

scene display means for displaying the representations 

of the objects in the viewer space; 

scene control means responsive to a scene control 
signal received from one of the first and second input means for 
manipulating the objects in the object space and/or moving the 
viewer space relative to the object space; 

system control display means for presenting in a 
two-dimensional form a number of different system control 
options ; 

selection control means responsive to a cursor control 
signal and a selection control signal, received from the other of 
the first and second input means and the signalling means 
respectively, by means of which selection control means the user 
can point to and select a desired one of the system control 
options . 

Input devices of the type set forth in the opening paragraph 
come in various forms, common examples being tracker balls, 
joysticks, and simple arrays of two or more potentiometers. Such 
multi-dimensional input devices are of particular use as the 
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scene control input device in interactive display systems as set 
forth in the second opening paragraph which are known and used in 
the fields of computer-aided design (GAD), education and 
entertainment - 

The cursor control means forms part of a menu-and-pointer 
type user interface for higher-level control and may conveniently 
but not necessarily share a common display device with the 3-D 
scenes. An input device as set forth in the opening paragraph 
could also be used as the input means for cursor control. 
Alternatively, the input means for cursor control might comprise 
for example a so-called "mouse" or a digitising tablet. A common 
problem in either case is that the table-top space required for 
the mouse or other cursor control device conflicts with the space 
occupied by the tracker-ball or other scene control device, and 
the user is inconvenienced greatly by having to move his/her hand 
continually between two different devices. 

A known solution to these problems is to provide means for 
switching the system between two modes so that a common input 
device can act in one mode (scene control) to control the various 
3-D manipulations and in another mode (cursor control) to control 
the movements of the menu pointer and so on for higher level 
control. A disadvantage with this known solution is that a 
signalling button on the input device is permanently occupied (at 
least in the scene control mode) with the mode-switching 
activity. Another problem is that the user commonly forgets 
which mode the device is in, and commits errors which even at the 
least are a waste of time and a source of irritation to the 
user. 

It is an object of the invention to provide a user input 
device with which some or all of the above disadvantages can be 
overcome . 

The invention provides a user input device for an 
interactive display apparatus, the device comprising a body on 
which are mounted first input means by which hand movements of a 
user relative to the body are translated into a first control 
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signal variable in at least two dimensions, characterised in that 
the device further comprises second input means by which 
movement by the user of the body in two dimensions across a 
reference surface against which the btfdy rests is translated into 
5 a second control signal also variable in at least two 

dimensions. In such a device, two input means are combined in a 
single unit and yet they are able to operate independently 
without the need for mode-switching and without the risk of 
confusion arising therefrom. 
10 T he second input means in such a device is useful 

particularly, but not exclusively, for cursor control since 
movement across a two-dimensional surface (such as a table-top) 
can be related very easily in the mind of the user to resulting 
movements of a cursor over a two-dimensional screen display. 
15 The second input means may also be mounted in the body and 

may comprise a ball protruding partially through an opening in 
the body, which ball can be rotated relative to the body by 
frictional contact with the reference surface, and means for 
detecting such rotation with respect to two orthogonal axes fixed 
20 with respect to the body and substantially parallel to the 

reference surface when the device is in use. This form of input 
means can be the same as that used in the conventional mechanical 
type of mouse input device, giving a simple and low-cost 
construction and compatibility with an existing mouse interface. 
25 Alternatively, the second input means may comprise optical 

sensing means for detecting movement of the body using reference 
marks provided on the reference surface. This second input means 
is similar to a conventional electro-optical type of mouse in 
which there are no moving parts and a particularly compact 
construction can be obtained. 

•The device may further comprise signalling means located on 
the body whereby a user can with a single hand move the body to 
alter one or both of the first and second control signal and at 
the same time generate selection control signals to initiate 
further processing of selected values of the first and/or second 
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control signal- Signalling means, for example one or more 
push-switch.es, can allow selections of display items pointed to 
by a cursor, mode changes and other useful facilities, all under 
control of a single hand. Where signalling means are provided 
5 for use in association with the second input means, the 

signalling means may comprise at least one switch located on a 
part of' the body which in use is inclined or substantially 
perpendicular to the reference surface. This may avoid the 
positioning of the signalling means interfering with that of the 
10 first input means, while enabling the switch(es) to be positioned 
within reach of the user's fingers while the body is being moved 
over the surface to operate the second input means. 

In principle 3 any form of two-or-more dimensional input 
device may be mounted on the body to form the first input means. 
In some embodiments, the first input means comprises a control 
element projecting through an aperture in the body so as to 
enable the generation of the first control signal in response to 
hand movement in two dimensions relative to the body. The 
control element may for example comprise a joystick, the first 
input means comprising detectors responsive to deflection of the 
joystick in two orthogonal directions. It should be noted, 
however, that substantial movement of the control element is not 
essential; it may be sufficient to detect the force of hand 
movements without the control element itself moving. 

In another such embodiment the control element of the first 
input means comprises a ball, rotatably mounted within the body 
and partially protruding through an aperture in the body and the 
first input means comprises detectors for detecting rotation of 
the ball about two orthogonal axes to provide first and second 
dimensions to the first control signal. 

In such embodiments, the first input means may further 
comprise a rotatable element surrounding or partially surrounding 
the aperture, and a further detector for detecting rotation of 
the element around the periphery of the aperture to provide a 
35 third dimension to the first control signal. The rotatable 
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element may comprise a ring surrounding the aperture. 

The invention further provides an interactive display system 
of the type set forth in the second opening paragraph, 
characterised in that the first and second input means and the 
signalling means are formed by a user input device in accordance 
with an embodiment of the invention as set forth above. 

In particular, the scene control means may be responsive to 
a control signal generated by the first input means and the 
cursor control means are responsive to a cursor control signal 
generated by the second input means. 

Embodiments of the invention will now be described, by way 
of example, with reference to the accompanying drawings, in 
which: 

Figure 1 shows schematically an interactive display 
system including first and second input means; 

Figure 2 is a cross-sectional view of through an input 
device in accordance with one embodiment of the invention; 

Figure 3 is a view from below of an upper part of the input 

device of Figure 2; and 

Figure 4 is a view from above of a lower part of the input 

device of Figures 2 and 3; 

Figure 5 is an external perspective view of part of the 
input device of Figures 2 to 4; and 

Figure 6 is an external perspective view of a similar part 
of an input device in accordance with another embodiment of the 
invention . 

Figure 1 shows schematically a system for the display of 
images representing 3-D objects. A group 10 of modules comprises 
a module OBJ for storing and managing a database forming a model 
of a 3-D environment, the "object space", containing various 3-D 
objects, each modelled for example as a group of polygonal 
surfaces, which in turn are defined by vertex coordinates and 
surface characteristics (colour, texture, transparency etc.). 
Positions and other attributes of light sources may also be 
defined. Another module VEW defines a "viewing space-', for 
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example in terms of the position of a viewpoint, the direction of 
view and the field of view. 

A scene control input means 12 provides control in two or 
more directions simultaneously so that the user can manipulate or 
modify objects within the object space or manipulate the 
viewpoint, direction of view and other attributes of the viewing 
space. The input means 12 comprises transducers for converting 
the user's hand movements relative to a body of the input device 
into signals which can be interpreted by an interface module 
3-DIN. Many forms of two-dimensional input means are known for 
this purpose, including tracker balls, joysticks, electrostatic, 
electroacoustic and electromagnetic digitising tablets, and many 
of these can be extended with controls such as potentiometers for 
further degrees of freedom, and with signalling buttons 14 for 
mode-changing commands. Another form of input device known as a 
"knob box" comprises a box on which is mounted an array of 
potentiometers, one for each degree of freedom which it is 
desired to control, often as many as nine. 

A module TRANS performs geometric transformations to 
translate the various coordinates in object space defined by the 
module OBJ to equivalent coordinates in the viewing space. These 
transformations will generally include translation, rotation and 
perspective, as is well-known in the art. In the case where the 
display is only two-dimensional coordinates in the viewing space 
25 in two dimensions (typically called x and y) will correspond 
linearly to screen coordinates, while a third coordinate (z) 
corresponds to the "depth" of an object "into" the screen. 

The module TRANS passes the viewing space coordinates and 
surface attributes to a rendering module DRW which converts the 
30 viewer space information into a form suitable for display, 

typically by scan conversion into pixel values in a bit-mapped 
display memory DIS. The rendering module DRW may perform 
calculations for hidden-surface removal (using the z 
coordinates), lighting, shading, anti-aliasing and clipping to 
35 the screen boundaries in a manner well-known in the art. 
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The display memory DIS is mapped onto a window area 16 of a 
display screen 18. A control module HCI uses the remainder of 
the screen 18 to display menus of control options, and receives 
position signals from a cursor control input means 20 which 
enable the display of a cursor 22 at locations selected by the 
user. The cursor control input means 20 could comprise any of 
the two-dimensional input devices mentioned above as being 
suitable for use as the scene control input means 12. However, 
it is also possible to use a kind of device which is movable 
bodily relative to a stationary reference to generate the 
required signals. Examples of such devices are the mechanical 
and optical types of "mouse", which detect movements of their 
bodies over a relatively passive reference surface such as a 
table top. Other such devices are the electromagnetic, 
electroacoustic and electrostatic types of digitising tablet, 
where the reference surface forms a more an active component. 
These devices are particularly convenient for pointing by cursor 
movement, since the user can readily equate hand movements over a 
flat surface to the resulting movements of the cursor over the 
screen. A type of input device suitable for cursor control and 
which responds to hand movements relative to the device body xs 
described in European Patent Application EP-A2-0320044. Such 
devices may also be used as the scene control input means 12 xn 
piace of the devices mentioned above in that context. Signaling 
means 24, typically comprising one or more push button switches, 
are also mounted on the body so that the user can indicate that a 
menu option (26) to which the cursor 22 is pointing when the 
button is pressed is being selected. 

The menu options displayed may enable various global 
parameters of the object space or the viewing space to be 
adjusted. "Sliders" displayed (at 28, for example) may be 
convenient for this purpose and also, for example to allow the 
altitude of the viewpoint to be adjusted if this is not possible 
using the scene control input means 12. Other options may allow 
the creation or addition of new 3-D objects, a change to a 
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different model, file management functions and so forth. It will 
be understood that the significance of the signals generated by 
scene control input means 12 can thus vary in accordance with 
different modes of operation activated using the cursor control 

5 input means 20. 

The various elements of the system illustrated as described 
so far and their functional relationships are conventional, and 
can remain conventional in a system in accordance with the 
invention. However, the problems created for the user in 

10 manually operating both input means 12 and 20 can be reduced in 

accordance with the invention by adopting a novel construction of 
input device in which both input means 12 and 20 are combined in 
a single unit. One such device is illustrated in Figures 2, 3 
and 4. 

15 Figure 2 shows the device in cross-section. A housing of 

the device comprises upper and lower body parts 30 and 32 
respectively. The face 34 of the lower body part 32 which faces 
towards the left hand side of Figure 2 rests on a table top or 
other reference surface when the device is in use. 

Describing the upper body part 30 first (Figures 2 and 3), 
this houses a first input means including a control element in 
the form of a tracker ball 36 and an associated mechanical and 
opto-electronic sub-assembly 38. The construction of such a 
sub-assembly is well-known. In essence it comprises two rollers 
with orthogonal axes biased against the surface of the ball 36 so 
as to rotate when the user rotates the ball about a corresponding 
axis by pushing against the part 40 of the ball's surface which 
protrudes through the body part 30. One of the rollers can be 
seen in cross-section at 42. Optical encoders translate the 
roller rotations into movement signals which are transmitted to 
the display apparatus via a cable 44 which leaves the device in a 
direction away from the user. A resting area 45 is provided to 
take the weight of the hand while the fingers manipulate the 
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For convenience, the tracker ball 36 and sub-assembly 38 
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could be taken from a commercially available tracker ball input 
device. One such is available in the United Kingdom from 
RS Components Limited under the Stock number 650-964. Other 
types of first input means such as those already mentioned could 
be provided instead of the tracker ball assembly, and could 
likewise be taken from a suitable commercially available devxce. 
A joystick mechanism for example is available commercially from 
KS Components Limited under Stock Number 162-732. 

The commercially available tracker ball assembly 36/38 has 
been enhanced in the device shown to provide a third degree of 
control. To this end, a bezel 46 surrounding the ball 36 where 
it protrudes through the body part 30 is journalled so as to be 
rotatable about an axis 48 which is orthogonal to the roller axes 
about which the ball rotation is detected by the rollers. The 
bezel is grooved to receive and engage a drive belt 50 which 
connects it to a pulley 52 which in turn drives the shaft 54 of 
an optical encoder assembly 56. Such encoder may conveniently be 
of the same form as those in the tracker ball or mouse. The bezel 
46 can be rotated conveniently by the same hand that controls the 
ball 36, in response to which signals are generated by the 
encoder 56 and transmitted via the cable 44 to the display 
apparatus. Wireless transmission rather than cable transmission 
WO uld also be possible of course, for example by a modulated 
infra-red signal. It may be noted that all signals transmxtted 
by the device to the display apparatus may be of a conventional 
form, so that interfacing with existing display apparatus is not 

a problem. . . _ 

Figure 5 shows part of the exterior of the device xncludxng 
the tracker ball 36 and the rotating bezel 46. It will be seen 
that a slight opening of the fingers of a user's hand will put 
them in a position for moving the bezel 46 while the main part of 
the hand remains substantially unmoved from its conventional 
position over the ball 36. The bezel 46 may be knurled on its 
top surface to provide a good grip. Further degrees of control 
could be provided by further rotary control elements concentrxc 
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with the bezel 46. 

This enhancement is also applicable to other types of input 
device where a control element projects from an aperture in a 
housing so as to be movable by a user's hand, and is the subject 
of United Kingdom Patent Application No. «2 ? \~7-~54r2- 
having the same application date as the present application. In 
the embodiment of the invention illustrated in Figure 6, a 
ring-shaped control element 90 is mounted so as to rotate around 
the aperture 92 through which a control element in the form of a 
joystick 94 projects, again to provide a third dimension of 
control in addition to the conventional joystick's two control 
axes. It may be noted that it is known to enhance a joystick by 

v a* ai- the end o* the stick rotatable about the 
making a knoo ju »t — — — - 

joystick axis to provide a third degree of control. The rotating 
ring 90 may be provided as an alternative to the known 
enhancement, or may be used in addition. 

On the lower body part 32 (Figures 2 and 4) there are 
mounted mechanical and electronic sub-assemblies 60 and 62 of a 
second input means similar to the conventional mouse. A mouse 
ball 64 projects through the base 34 of the device and rolls over 
a table top or other surface upon which the device is resting in 
use. A spring arrangement 66 biases the ball 64 against two 
orthogonal rollers 68 and 70 which rotate as the ball 64 is 
rolled across the table top in two corresponding orthogonal 
directions. A pad 80 glued to the inside of the upper body part 
30 prevents the ball falling out of place, while a sheet of felt 
82 on the underside of the body allows it to slide smoothly over 
a table surface. Each roller 68,70 drives a respective optical 
interruptor wheel 72,74 and optical encoders 76 and 78 generate 
movement signals which are transmitted with the signals from the 
first input means via the cable 44. The sub-assembly 38 for the 
tracker ball 36, although not shown in detail, is substantially 
equivalent in operation to that of the mouse as just described. 

To indicate selections from a menu and so forth, a cursor 
control input means such as a mouse is normally associated with 
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some signalling means such as one or more push button switches, 
conventionally located on top of the body of the mouse, where the 
user's fingers can operate them while moving the mouse across the 
table under the palm of the hand. However the top side of the 
present device is occupied by the tracker ball, and will tend to 
be similarly occupied even in other embodiments. The tracker 
ball or other first input means may also have associated 
signalling buttons which do need to be mounted on the top side of 
the housing, so that using this location for signalling means 
associated with the second input means could lead to confusion. 
The device therefore has signalling buttons 84 and 86 mounted on 
the inclined side walls of the upper body part 30 of the device 
housing. 

These positions are particularly advantageous since when the 
user wishes to operate the second input means as a cursor control 
input means for menu selection and so forth, the hand will 
naturally shift from a forward position above the tracker ball to 
a position gripping the body in the region of the resting area 
45, with the fingers and thumb extending down over the opposite 
inclined sides. Signals from the pushbuttons 84 and 86 may be 
transmitted to the display apparatus via the cable 44 in a 
conventional manner. 

The mouse sub-assemblies 60 and 62 and the switches 84 and 
86 can conveniently be taken from a commercially available 
mouse. One such is available in the United Kingdom from 
RS Components Limited under Stock number 650-942. Another is 
described in united States Patent No. 4 464 652. However, other 
types of mouse or other devices such as digitising tablet cursors 
could be incorporated into the device to the same effect. 

One such variation which might be of benefit in the combined 
device is the so-called optical mouse, such as those described in 
United States Patents Nos. 4 346 035 and 4 546 347, the contents 
of which are incorporated herein by reference. The sub-assembly 
for an optical mouse can be made more compact than that of a 
mechanical mouse, since the ball and other moving parts are 



10 



15 



20 



25 



30 



12 PHB3*575 

replaced by a purely electro-optical system. This enables the 
complete device to be constructed in a housing very little larger 
than that required for the first input means alone. A drawback 
of the optical mouse technique is that the reference surface has 
to be marked with a special coloured grid. 

From reading the present disclosure, other variations will 
be apparent to persons skilled in the art. Such variations may 
involve other features which are already known in the design, 
manufacture and use of display systems, user input devices and 
component parts thereof and which may be used instead of or in 
addition to features already described herein. Although claims 
have been formulated in this application to particular 
combinations of features, it should be understood that the scope 
of the disclosure of the present application also includes any 
novel feature or any novel combination of features disclosed 
herein either explicitly or implicitly or any generalisation 
thereof, whether or not it relates to the same invention as 
presently claimed in any claim and whether or not it mitigates 
any or all of the same technical problems as does the present 
invention. The applicants hereby give notice that new claims may 
be formulated to such features and/or combinations of such 
features during the prosecution of the present application or of 
any further application derived therefrom. 
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CLAIMS : 

1. A user input device for an interactive display 
apparatus, the device comprising a body on which are mounted 
first input means by which hand movements of a user relative to 

5 the body are translated into a first control signal variable in 

at least two dimensions, characterised in that the device further 
comprises second input means by which movement by the user of the 
body in two dimensions across a reference surface against which 
the body rests is translated into a second control signal 

1° variable in two dimensions. 

2. A device as claimed in Claim 1 wherein the second input 
means is also mounted in the body and comprises a ball protruding 
partially through an opening in the body, which ball can be 
rotated relative to the body by frictional contact with the 

15 reference surface, and means for detecting such rotation with 

respect to two orthogonal axes fixed with respect to the body and 
substantially parallel to the reference surface when the device 

is in use. , 

A device as claimed in Claim 1 wherein the second input 
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mea ns comprises optical sensing means for detecting movement of 
the body using reference marks provided on the reference surface. 

4 A device as claimed in any of Claims 1 to 3 further 
comprising signalling means located on the body whereby a user 
can with a single hand alter one or both of the first and second 
control signals and at the same time generate selection control 
signals to initiate further processing of selected values of the 
first and/or second control signal. 

5 A device as claimed in Claim 5 wherein the signalling 
means comprises at least one switch located on a part of the body 
which in use is inclined or substantially perpendicular to the 

reference surface. 

6. A device as claimed in Claim 5 wherein the at least one 
switch' comprises a push switch operable by a user pressing a 
finger against the inclined part of the body. 

7. A device as claimed in any of Claims 1 to 6 wherein the 
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first input means comprises a control element projecting through 
an aperture in the body so as to enable the generation of the 
first control signal in response to hand movement in two 
dimensions relative to the body, 
5 8. A device as claimed in Claim 7 wherein the control 

element of the first input means comprises a joystick projecting 
from the body and the first input means comprises detectors 
responsive to deflection of the joystick in two orthogonal 
directions . 

10 9. A device as claimed in Claim 7 wherein the control 

element of the first input means comprises a ball rotatably 
mounted within the body and partially protruding through an 
aperture in the body and the first input means comprises 
detectors responsive to rotation of the ball about two orthogonal 

15 axes to provide first and second dimensions to the first control 

signal - 

10. A device as claimed in any of Claims 7 to 9 wherein the 
first input means further comprises a rotatable element 
surrounding or partially surrounding the aperture, and a further 

20 detector for detecting rotation of the said rotatable element 

around the periphery of the aperture to provide a third dimension 
to the first control signal. 

11. A device as claimed in Claim 10 wherein the rotatable 
element comprises a ring surrounding the aperture. 

25 12. A user input device substantially as described herein 

with respect to the accompanying drawings • 

13. An interactive display system comprising: 

- first input means, second input means and signalling 
means ; 

30 - three-dimensional (3-D) modelling means for defining a 

set of objects in a 3-D object space; 

- transformation means for generating representations of 
the objects in a viewer space; 

- scene display means for displaying the representations 

35 , 

of the objects in the viewer space; 
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- scene control means responsive to a scene control 
signal received from one of the first and second input means for 
manipulating the objects in the object space and/or moving the 
viewer space relative to the object space; 

5 - system control display means for presenting in a two 

dimensional form a movable cursor and a number of different 
system control options; 

selection control means responsive to a cursor control 
signal and a selection control signal, received from the other of 

10 the first and second input means and the signalling means 

respectively, by means of which selection control means the user 
can point to and select a desired one of the system control 
options ; 

characterised in that the first and second input means and the 
signalling means are formed by a user input device as claimed in 
any of Claims 5, 6 or 7 . 

14. An interactive display system as claimed in Claim 14 
wherein the scene control means are responsive to a scene control 
signal generated by the first input means and the selection 
control means are responsive to a cursor control signal received 
from the second input means . 

15. An interactive display system substantially as described 
herein with reference to the accompanying drawings. 
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